Distinguishing lane markings for a vehicle to follow

ABSTRACT

The present invention extends to methods, systems, and computer program products for distinguishing lane markings for a vehicle to follow. Automated driving or driving assist vehicles utilize sensors to help the vehicle navigate on roadways or in parking areas. The sensors can utilize, for example, the painted surface markings to help guide the vehicle on its path. Aspects of the invention use a first type of sensor and at least a second different type of sensor to identify road surface markings. When ambiguity is detected between road surface markings, decision making algorithms identify the correct set of markings for a vehicle to abide by. The sensors also identify the location and trajectory of neighboring vehicles to increase confidence with respect to the identified road-surface markings.

CROSS-REFERENCE TO RELATED APPLICATIONS

Not applicable.

BACKGROUND

1. Field of the Invention

This invention relates generally to the field of road line detectionsystems, and, more particularly, to road line detection systems whichcan distinguish lane markings for a vehicle to follow.

2. Related Art

Many industries, including the automotive industry, are utilizingadvanced automation capabilities. The opportunity exists to automatemany of the functions of a traditional vehicle. Some research is beingperformed in the automotive industry to make a vehicle completelyautonomous; requiring no human input at all.

Several benefits exist as the result of vehicles becoming more automatedor completely autonomous. Benefits include reduced traffic accidents,improved traffic flow, and reduced fuel consumption, just to name a few.For example, many vehicles are utilizing collision avoidance technologywhere the vehicle's computer system can apply the brakes if thevehicle's sensors detect that the vehicle is in danger of colliding withanother vehicle or object. Also, many vehicles have introduced aparallel parking feature which allows the vehicle to parallel parkwithout any input from the driver of the vehicle.

The plausibility of automated or autonomous vehicles is heavilydependent on the ability of the vehicle sensors and the vehicle computersystem to identify and process the vehicle's environment and to react todifferent situations in a safe and efficient manner. However, thevehicle environment is very dynamic. Thus, the vehicle computer systemneeds to be able to perform in varying weather conditions, such as, rainor snow, and react to various road conditions, such as, icy roads orroads that are under construction. The challenge exists to guarantee asafe and efficient driving experience, regardless of the vehicleenvironment.

Some vehicle automation techniques used for vehicle navigation considerthe painted lane markings on the surface of a road. The lane markingscan be used as a means to safely guide the vehicle along its route.However, at least a few difficulties arise from the use of lanemarkings. For example, road surface markings not only include lanemarkings, but they also include navigation information, such as, turninglane indicators, speed limit information, crosswalk information,railroad information, and high-occupancy vehicle (HOV) markings.Distinguishing between lane markings and other road surface markings canbe difficult.

Another challenge is distinguishing between ambiguous lane markers thatcan occur, for example, during road construction, maintenance, orupgrades. Lanes can be shifted to one side or another during periods ofconstruction to facilitate the maintenance of roads. During the periodof construction road maintenance, or upgrades, new lane markings arepainted onto the road and old lane markings can either be scraped awayor painted in the color of the road in an effort to mask the old lanemarkings. However, masking attempts are not always fully successful andold lane markings can sometimes be seen after scraping or re-painting.As such, conditions may exist where two sets of lane markings can beseen for a particular lane of the road. Determining which set of lanemarkings to follow in an automated manner can be difficult.

BRIEF DESCRIPTION OF THE DRAWINGS

The specific features, aspects and advantages of the present inventionwill become better understood with regard to the following descriptionand accompanying drawings where:

FIG. 1 illustrates an example block diagram of a computing device.

FIG. 2 illustrates an example computer architecture that facilitatesdistinguishing lane markings for a vehicle to follow.

FIG. 3 illustrates a flow chart of an example method for distinguishinglane markings for a vehicle to follow.

FIG. 4 illustrates an example roadway scenario.

DETAILED DESCRIPTION

The present invention extends to methods, systems, and computer programproducts for distinguishing lane markings for a vehicle to follow.

Embodiments of the present invention may comprise or utilize a specialpurpose or general-purpose computer including computer hardware, suchas, for example, one or more processors and system memory, as discussedin greater detail below. Embodiments within the scope of the presentinvention also include physical and other computer-readable media forcarrying or storing computer-executable instructions and/or datastructures. Such computer-readable media can be any available media thatcan be accessed by a general purpose or special purpose computer system.Computer-readable media that store computer-executable instructions arecomputer storage media (devices). Computer-readable media that carrycomputer-executable instructions are transmission media. Thus, by way ofexample, and not limitation, embodiments of the invention can compriseat least two distinctly different kinds of computer-readable media:computer storage media (devices) and transmission media.

Computer storage media (devices) includes RAM, ROM, EEPROM, CD-ROM,solid state drives (“SSDs”) (e.g., based on RAM), Flash memory,phase-change memory (“PCM”), other types of memory, other optical diskstorage, magnetic disk storage or other magnetic storage devices, or anyother medium which can be used to store desired program code means inthe form of computer-executable instructions or data structures andwhich can be accessed by a general purpose or special purpose computer.

A “network” is defined as one or more data links that enable thetransport of electronic data between computer systems and/or modulesand/or other electronic devices. When information is transferred orprovided over a network or another communications connection (eitherhardwired, wireless, or a combination of hardwired or wireless) to acomputer, the computer properly views the connection as a transmissionmedium. Transmissions media can include a network and/or data linkswhich can be used to carry desired program code means in the form ofcomputer-executable instructions or data structures and which can beaccessed by a general purpose or special purpose computer. Combinationsof the above should also be included within the scope ofcomputer-readable media.

Further, upon reaching various computer system components, program codemeans in the form of computer-executable instructions or data structurescan be transferred automatically from transmission media to computerstorage media (devices) (or vice versa). For example,computer-executable instructions or data structures received over anetwork or data link can be buffered in RAM within a network interfacemodule (e.g., a “NIC”), and then eventually transferred to computersystem RAM and/or to less volatile computer storage media (devices) at acomputer system. RAM can also include solid state drives (SSDs or PCIxbased real time memory tiered Storage, such as FusionIO). Thus, itshould be understood that computer storage media (devices) can beincluded in computer system components that also (or even primarily)utilize transmission media.

Computer-executable instructions comprise, for example, instructions anddata which, when executed at a processor, cause a general purposecomputer, special purpose computer, or special purpose processing deviceto perform a certain function or group of functions. The computerexecutable instructions may be, for example, binaries, intermediateformat instructions such as assembly language, or even source code.Although the subject matter has been described in language specific tostructural features and/or methodological acts, it is to be understoodthat the subject matter defined in the appended claims is notnecessarily limited to the described features or acts described above.Rather, the described features and acts are disclosed as example formsof implementing the claims.

Those skilled in the art will appreciate that the invention may bepracticed in network computing environments with many types of computersystem configurations, including, personal computers, desktop computers,laptop computers, message processors, hand-held devices, multi-processorsystems, microprocessor-based or programmable consumer electronics,network PCs, minicomputers, mainframe computers, mobile telephones,PDAs, tablets, pagers, routers, switches, various storage devices, andthe like. The invention may also be practiced in distributed systemenvironments where local and remote computer systems, which are linked(either by hardwired data links, wireless data links, or by acombination of hardwired and wireless data links) through a network,both perform tasks. In a distributed system environment, program modulesmay be located in both local and remote memory storage devices.

Embodiments of the invention can also be implemented in cloud computingenvironments. In this description and the following claims, “cloudcomputing” is defined as a model for enabling ubiquitous, convenient,on-demand network access to a shared pool of configurable computingresources (e.g., networks, servers, storage, applications, and services)that can be rapidly provisioned via virtualization and released withminimal management effort or service provider interaction, and thenscaled accordingly. A cloud model can be composed of variouscharacteristics (e.g., on-demand self-service, broad network access,resource pooling, rapid elasticity, measured service, etc.), servicemodels (e.g., Software as a Service (SaaS), Platform as a Service(PaaS), Infrastructure as a Service (IaaS), and deployment models (e.g.,private cloud, community cloud, public cloud, hybrid cloud, etc.).Databases and servers described with respect to the present inventioncan be included in a cloud model.

Further, where appropriate, functions described herein can be performedin one or more of: hardware, software, firmware, digital components, oranalog components. For example, one or more application specificintegrated circuits (ASICs) can be programmed to carry out one or moreof the systems and procedures described herein. Certain terms are usedthroughout the following description and Claims to refer to particularsystem components. As one skilled in the art will appreciate, componentsmay be referred to by different names. This document does not intend todistinguish between components that differ in name, but not function.

In general, aspects of the invention are directed to distinguishing lanemarkings for a vehicle to follow. At least two different types ofsensory devices gather data related to the vehicles environment. Thedata is processed to identify road surface markings and neighboringvehicles. Important regions of interest are extracted from the processeddata. Lane marking data and neighboring vehicle data is utilized toidentify the correct set of lane markings for a vehicle to follow. Thecorrect lane markings are utilized by the vehicle computer system andcontrol system to navigate the vehicle on the correct lane of the road.

FIG. 1 illustrates an example block diagram of a computing device 100.Computing device 100 can be used to perform various procedures, such asthose discussed herein. Computing device 100 can function as a server, aclient, or any other computing entity. Computing device 100 can performvarious communication and data transfer functions as described hereinand can execute one or more application programs, such as theapplication programs described herein. Computing device 100 can be anyof a wide variety of computing devices, such as a mobile telephone orother mobile device, a desktop computer, a notebook computer, a servercomputer, a handheld computer, tablet computer and the like.

Computing device 100 includes one or more processor(s) 102, one or morememory device(s) 104, one or more interface(s) 106, one or more massstorage device(s) 108, one or more Input/Output (I/O) device(s) 110, anda display device 130 all of which are coupled to a bus 112. Processor(s)102 include one or more processors or controllers that executeinstructions stored in memory device(s) 104 and/or mass storagedevice(s) 108. Processor(s) 102 may also include various types ofcomputer storage media, such as cache memory.

Memory device(s) 104 include various computer storage media, such asvolatile memory (e.g., random access memory (RAM) 114) and/ornonvolatile memory (e.g., read-only memory (ROM) 116). Memory device(s)104 may also include rewritable ROM, such as Flash memory.

Mass storage device(s) 108 include various computer storage media, suchas magnetic tapes, magnetic disks, optical disks, solid state memory(e.g., Flash memory), and so forth. As depicted in FIG. 1, a particularmass storage device is a hard disk drive 124. Various drives may also beincluded in mass storage device(s) 108 to enable reading from and/orwriting to the various computer readable media. Mass storage device(s)108 include removable media 126 and/or non-removable media.

I/O device(s) 110 include various devices that allow data and/or otherinformation to be input to or retrieved from computing device 100.Example I/O device(s) 110 include cursor control devices, keyboards,keypads, barcode scanners, microphones, monitors or other displaydevices, speakers, printers, network interface cards, modems, cameras,lenses, CCDs or other image capture devices, and the like.

Display device 130 includes any type of device capable of displayinginformation to one or more users of computing device 100. Examples ofdisplay device 130 include a monitor, display terminal, video projectiondevice, and the like.

Interface(s) 106 include various interfaces that allow computing device100 to interact with other systems, devices, or computing environmentsas well as humans. Example interface(s) 106 can include any number ofdifferent network interfaces 120, such as interfaces to personal areanetworks (PANs), local area networks (LANs), wide area networks (WANs),wireless networks (e.g., near field communication (NFC), Bluetooth,Wi-Fi, etc., networks), and the Internet. Other interfaces include userinterface 118 and peripheral device interface 122.

Bus 112 allows processor(s) 102, memory device(s) 104, interface(s) 106,mass storage device(s) 108, and I/O device(s) 110 to communicate withone another, as well as other devices or components coupled to bus 112.Bus 112 represents one or more of several types of bus structures, suchas a system bus, PCI bus, IEEE 1394 bus, USB bus, and so forth.

Aspects of the invention can be used to identify and compare roadsurface markings in order to correctly abide by relevant (e.g., newer)road instructions. Camera, lidar (light detection and ranging), andother range sensors can be used to capture information of the road andsurrounding vehicles. Computer vision and sensor fusion algorithmsemploying neural networks can be trained to recognize surroundingvehicles and conflicting road surface marking regions of interest onroadways and parking lots. When multiple sets of markings are detected,factors including intensity of color, reflectivity, presence of lanereflectors, paint cracking, and road tape peeling can be used to comparesurface marking sets and determine relevant (e.g., newer) markings tofollow. Other roadway marking information can also be used. The behaviorof other vehicles can also be used to add certainty when determining aset of markings to follow.

Identifying and comparing road surface markings can include: (1) Usingneural networks to identify surface markings on the road and othervehicles within sensor viewing ranges, (2) Determining bounding boxesfor identified regions of interest, (3) Sending image data withinbounding boxes to algorithms configured to determine if there aremultiple sets of road surface markings, and if so, compare the sets todetermine the relevant set of markings to follow, and (4) Sending therelevant road surface markings to the vehicle decision makingalgorithms.

FIG. 2 illustrates an example computer architecture 200 that facilitatesdistinguishing lane markings for a vehicle to follow. Computerarchitecture 200 can be contained within a vehicle, such as, forexample, a car, a truck, a bus, or a motorcycle. Referring to FIG. 2,computer architecture 200 includes computer system 201, image-capturedevice 210, and lidar system 220. In some embodiments, radar system 230is also included. Each of computer system 201, image-capture device 210,lidar system 220, and radar system 230, as well as their respectivecomponents can be connected to one another over (or be part of) anetwork, such as, for example, a PAN, a LAN, a WAN, and even theInternet. Accordingly, each of computer system 201, image-capture device210, lidar system 220, and radar system 230 as well as any otherconnected computer systems and their components, can create messagerelated data and exchange message related data (e.g., near fieldcommunication (NFC) payloads, Bluetooth packets, Internet Protocol (IP)datagrams and other higher layer protocols that utilize IP datagrams,such as, Transmission Control Protocol (TCP), Hypertext TransferProtocol (HTTP), Simple Mail Transfer Protocol (SMTP), etc.) over thenetwork.

In one aspect, each of image-capture device 210, lidar system 220, andpossibly radar system 230 are included in a vehicle. Each of the sensingdevices is configured to sense data in the vicinity of the vehicle. Forexample, image-capture device 210 is configured to capture image data212, lidar system 220 is configured to capture lidar data 222, and radarsystem 230 is configured to capture radar data 232.

As depicted, computer system 201 includes data processing module 240 andcontrol system module 250. Data processing module 240 is configured toreceive data from the sensing devices. For example, data processingmodule 240 can receive image data 212 from image-capture device 210,lidar data 222 from lidar system 220, and in some aspects, radar data232 from radar system 230.

Data processing module 240 further includes road surface markingidentification module 242, neighboring vehicle identification module244, region of interest module 246, and lane identifying module 248.

Road surface marking identification module 242 is configured to identifyroad markings in the vicinity of a vehicle from information contained inreceived sensor data. Information can include attributes, such as, forexample, color intensity, reflectivity, presence of lane reflectors,paint cracking, and road tape peeling. Other roadway marking informationcan also be used.

Neighboring vehicle identification module 244 is configured to detectneighboring vehicles from information contained in received sensor data.For example, neighboring vehicle information module 244 can detect thelocation and trajectory of neighboring vehicles. Detected vehicles maybe going in the same direction as a vehicle that includes image-capturedevice 210, lidar system 220, and possibly radar system 230 (e.g., inthe same lane as the vehicle or in a lane adjacent to the vehicle).Detected vehicles may also be travelling toward the automated vehicle(e.g., in oncoming traffic).

Region of interest module 246 is configured to identify lane markingdata for determining the correct lane markings to follow. For example,region of interest module 246 can identify lane marking data 260relative to a vehicle and exclude other road surface markings such asspeed limit information, HOV markings, and the like. Region of interestmodule 246 can also identify neighboring vehicle data 262 related to thelocation and trajectory of neighboring vehicles in relevant proximity toa vehicle.

Region of interest module 246 is configured to pass lane marking data260 and neighboring vehicle data 262 to lane identifying module 248.Lane identifying module 248 is configured to utilize the attributes ofthe lane marking data 260 to determine the relevant set of lane markingsto follow. For example, the attributes of the lane markings may includeintensity of color, reflectivity, presence of lane reflectors, paintcracking, and road tape peeling. Attributes of other roadway markingscan also be used. Lane identifying module 248 can utilize neighboringvehicle data 262 to determine the location and trajectory of neighboringvehicles. Once lane identifying module 248 has identified relevant lanemarkings to follow, lane identifying module can use the location andtrajectory of neighboring vehicles to increase the confidence level thatthe relevant lane markings are in fact the lane markings to follow.

Control system module 250 is configured to utilize the relevant lanemarkings to control the vehicle's location and trajectory.

FIG. 3 illustrates a flow chart of an example method 300 fordistinguishing lane markings for a vehicle to follow. Method 300 will bedescribed with respect to the components and data of computerarchitecture 200.

Method 300 includes accessing sensor data which has been gathered by aplurality of sensors, the plurality of sensors including a first type ofsensor and at least a second different type of sensor, the sensor dataindicating road-surface markings, the road-surface markings includingintensity of color and reflectivity, the sensor data also indicating thelocation and trajectory of neighboring vehicles (301). For example, avehicle can include a computer system 201, image-capture device 210,lidar system 220, and possibly also radar system 230. Other sensors arepossible, as well. Image-capture device 210 can capture image data 212,lidar system 220 can capture lidar data 222, and, when included, radarsystem 230 can capture radar data 232. Data processing module 240 canaccess image data 212, lidar data 222, and, when appropriate, radar data232. Collectively, image data 212, lidar data 222, and, whenappropriate, radar data 232 (hereinafter also referred to as the“accessed sensor data”) can indicate road surface markings andneighboring vehicles in the vicinity of the vehicle. The indication ofroad surface markings can include data for determining color intensityand reflectivity for the road surface markings. The indication ofneighboring vehicles can include data for determining location andtrajectory.

Road surface marking module 242 can utilize the accessed sensor data toidentify road surface markings on the road. For example, road surfacemarking module 242 can identify painted lane markings, turn lanemarkings, stop sign markings, speed limit markings, and railroadcrossing markings, just to name a few.

Neighboring vehicle identification module 244 can utilize the accessedsensor data to identify neighboring vehicles. For example, neighboringvehicle identification module 244 can identify the location andtrajectory of neighboring vehicles on the road with the vehicle.

Method 300 includes determining bounding boxes from the accessed sensordata, the bounding boxes representing regions of interest fordistinguishing road-surface markings and neighboring vehicles (302). Forexample, region of interest module 246 can determine bounding boxes fromimage data 212, lidar data 222, and, when appropriate, radar data 232.In one aspect, image data 212 depicts an image of the road and theassociated lane markings and vehicles on the road at the time of theimage capture. Region of interest module 246 can isolate the lanemarking data and neighboring vehicle data necessary for identifying thelane for the vehicle to follow.

For example, road markings in image data 212 can include two solidyellow lines, two faded yellow lines, a solid dashed white line, a fadeddashed white line, a solid white line, and a faded white line. The fadedlines are not as bright or as prominent as the other lines describedherein. Region of interest module 246 can utilize bounding boxalgorithms to determine the bounding box of the solid yellow lines, thebounding box of the faded yellow lines, the bounding box of the soliddashed white line, the bounding box of the faded dashed white line, thebounding box of the solid white line, and the bounding box of the fadedwhite line.

Region of interest module 246 can also isolate neighboring vehicle datafrom the accessed sensor data utilizing bounding boxes. The accessedsensor data can indicate other vehicles on the road. For example, theaccessed sensor data may depict a vehicle in the same lane, a vehicle inthe adjacent lane, an oncoming vehicle in the adjacent lane, and anoncoming vehicle in the far lane. Region of interest module 246 canutilize bounding box algorithms to determine any of: a bounding box of avehicle in the same lane, a bounding box of a vehicle in the adjacentlane, a bounding box of the oncoming vehicle in an adjacent lane, and abounding box of a vehicle in an oncoming far lane.

Method 300 includes processing the accessed sensor data within thebounding boxes to determine that multiple road-surface markings arepresent (303). For example, data processing module 240 can process theaccessed sensor data contained within the bounding boxes to determinethat multiple lane markings exist in the lane of the road that thevehicle is traveling. The multiple lane markings can include two solidyellow lines and two faded yellow lines. Data processing module 240 canrecognize the two sets of road-surface markings as ambiguous and/orconflicting and determine that multiple road-surface markings arepresent. Data processing module 240 can represent the two sets ofroad-surface markings in lane marking data 260.

Data processing module 240 can also represent neighboring vehiclesidentified in neighboring vehicle data 262. For example, neighboringvehicle data 262 can identify the location and trajectory of neighboringvehicles.

In response to determining that multiple road-surface markings arepresent, method 300 includes utilizing the sensor data to identifyroad-surface markings, from among the multiple road-surface markings,the vehicle is to follow (304). For example, lane identifying module 248can utilize lane marking data 260 to identify the road-surface markings,from among the multiple road-surface markings, the vehicle is to follow.Lane identifying module 248 can utilize the accessed sensor data tocompare the intensity of color, reflectivity, presence of lanereflectors, paint cracking, and road tape peeling of both the two solidyellow lines and the two faded solid yellow lines and the solid dashedwhite line and the faded dashed white line. Other attributes can becompared, as well. Given the accessed sensor data, the lane identifyingmodule 248 can identify the two solid yellow lines and the solid dashedwhite line as the relevant lane markings for the lane for which thevehicle is to follow.

In response to determining that multiple road-surface markings arepresent, method 300 includes using the location and trajectory ofneighboring vehicles to increase confidence with respect to theidentified road-surface markings (305). For example, lane identifyingmodule 248 can utilize neighboring vehicle data 262 to identify thelocation and trajectory of neighboring vehicles to increase confidencewith respect to identification of the two solid yellow lines and thesolid dashed white line as the relevant lane markings. Lane identifyingmodule 248 can determine that a vehicle is moving in the same directionand in the same lane as the vehicle. Lane identifying module 248 candetermine that a neighboring vehicle is moving in the same direction asthe vehicle in an adjacent lane. Furthermore, lane identifying module248 can identify other neighboring vehicles as vehicles going in theopposite direction and in the adjacent lanes of the vehicle. Given thelocation and trajectory information of the neighboring vehicles, thelane identifying module 248 has increased confidence that two solidyellow lines and the solid dashed white line are relevant.

In response to determining that multiple road-surface markings arepresent, method 300 includes sending the identified road-surfacemarkings to the vehicle's decision making algorithms for use incontrolling the vehicle's location and trajectory (306). For example,data processing module 240 can send the two solid yellow lines and thesolid dashed white line to control system module 250 for use incontrolling the vehicle's location and trajectory.

FIG. 4 illustrates an example roadway scenario 400. As depicted, road402 includes lanes 403, 404, 405, and 406. Vehicle 450 is traveling inlane 403 on road 402. Vehicle 450 includes a computer system (not shown)that is configured to control the vehicle 450 in an autonomous mode. Inone aspect, the computer system is similar to computer system 201. Tothis end, the computer system may use lane information, such as lanemarkings, to estimate the bounds of lane 403.

Vehicle 450 includes image-capture device 452, lidar unit 454, and/orradar unit 456. Other sensors are possible, as well. Vehicle 450 can usethe plurality of sensors to obtain information about lane 403. Forexample, the computer system can use image-capture device 452 and lidarunit 454 to sense lane markings of lane 403. In some aspects, radar unit456 may also be used to gather information about the environment ofvehicle 450.

The computer system can be configured to identify regions of interestfrom the accessed sensor data utilizing bounding boxes. For example, theaccessed data from image-capture device 452 may depict an image of theroad 402. The image can include associated lane markings and vehicles onthe road 402 at the time of the image capture. The computer system canisolate the lane marking data and neighboring vehicle data necessary foridentifying the bounds of lane 403 for vehicle 450 to follow.

The computer system can isolate road marking data from the accessedsensor data utilizing bounding boxes. For example, the road markings inthe accessed sensor data may include two solid yellow lines 410, twofaded yellow lines 412, a solid dashed white line 420, a faded dashedwhite line 422, a solid white line 430, and a faded white line 432. Thecomputer system can utilize bounding box algorithms to determine: abounding box 411 of the solid yellow lines 410, a bounding box 413 ofthe faded yellow lines 412, a bounding box 421 of the solid dashed whiteline 420, a bounding box 423 of the faded dashed white line 422, abounding box 431 of the solid white line 430, and a bounding box 433 ofthe faded white line 432.

The computer system can also isolate neighboring vehicle data from theaccessed sensor data utilizing bounding boxes. The accessed sensor datamay depict other vehicles on the road. For example, the accessed sensordata can indicate vehicle 460 in lane 403, vehicle 462 in lane 404, anoncoming vehicle 464 in lane 405, and an oncoming vehicle 466 in lane406. The computer system can utilize bounding box algorithms todetermine: a bounding box 461 for vehicle 460, a bounding box 463 forvehicle 462, a bounding box 465 for oncoming vehicle 464, and a boundingbox 467 for vehicle 466.

The computer system can process the accessed sensor data containedwithin the bounding boxes to determine that multiple lane markings existin the lane 403. For example, the road markings in the accessed sensordata within the bounding boxes may depict both two solid yellow lines410 and two faded yellow lines 412. The computer system can recognizethe two sets of road-surface markings as ambiguous and/or conflictingsets of lane markings. As such, the computer system determines thatmultiple road-surface markings are present.

In response to determining that multiple road-surface markings arepresent, the computer system can utilize the accessed sensor data toidentify relevant road-surface markings that vehicle 450 is to follow.For example, the computer system can utilize the accessed sensor data tocompare the intensity of color, reflectivity, presence of lanereflectors, paint cracking, and road tape peeling of the two solidyellow lines 410 and the two faded yellow lines 412. Similarly, thecomputer system can utilize the accessed sensor data to compare theintensity of color, reflectivity, presence of lane reflectors, paintcracking, and road tape peeling of the solid white dashed line 420 andthe faded dashed white line 422. Other factors may also be used. Basedon the accessed sensor data, the computer system can identify the twosolid yellow lines 410 and the solid dashed white line 420 as therelevant (and correct) lane markings bounding lane 403.

Additionally, the computer system is able to use the location andtrajectory of neighboring vehicles to increase confidence with respectto identification of the two solid yellow lines 410 and the solid dashedwhite line 420. For example, the accessed sensor data can indicate thelocation and trajectory of vehicle 460. The computer system candetermine that vehicle 460 is moving in lane 403 in the same directionas vehicle 450. The accessed sensor data can indicate the location andtrajectory of vehicle 462. The computer system 401 can determine thatvehicle 462 is moving in lane 404 in the same direction as vehicle 450.Furthermore, the computer system 401 can identify vehicle 464 andvehicle 466 going in the opposite direction of vehicle 450 in lanes 405and 406 respectively. Given the location and trajectory information ofthe neighboring vehicles, the computer system has increased confidencethat the two solid yellow lines 410 and the solid dashed white line 420were appropriately identified.

After the correct lane markings have been identified, the computersystem can pass the identified correct lane markings to vehicle 450'sdecision making algorithms for use in controlling the vehicle's locationand trajectory.

Although the components and modules illustrated herein are shown anddescribed in a particular arrangement, the arrangement of components andmodules may be altered to process data in a different manner. In otherembodiments, one or more additional components or modules may be addedto the described systems, and one or more components or modules may beremoved from the described systems. Alternate embodiments may combinetwo or more of the described components or modules into a singlecomponent or module.

The foregoing description has been presented for the purposes ofillustration and description. It is not intended to be exhaustive or tolimit the invention to the precise form disclosed. Many modificationsand variations are possible in light of the above teaching. Further, itshould be noted that any or all of the aforementioned alternateembodiments may be used in any combination desired to form additionalhybrid embodiments of the invention.

Further, although specific embodiments of the invention have beendescribed and illustrated, the invention is not to be limited to thespecific forms or arrangements of parts so described and illustrated.The scope of the invention is to be defined by the claims appendedhereto, any future claims submitted here and in different applications,and their equivalents.

What is claimed:
 1. A method for automated control of a vehicle based onroadway surface lane markings, the method comprising a processor:accessing sensor data from a plurality of sensors at the vehicle, theplurality of sensors monitoring a roadway where the vehicle istraveling; determining bounding boxes from the accessed sensor data, thebounding boxes representing regions of interest in the roadway fordistinguishing roadway surface lane markings and neighboring vehicles;processing the accessed sensor data within the bounding boxes toidentify multiple sets of roadway surface lane markings; utilizing thesensor data to identify a set of roadway surface lane markings, fromamong the multiple sets of roadway surface lane markings, the vehicle isto follow; and controlling the vehicle within a lane of the roadway byfollowing the set of roadway surface lane markings.
 2. The method ofclaim 1, further comprising: processing the accessed sensor data toidentify location and trajectory of neighboring vehicles; and using thelocation and trajectory of neighboring vehicles to increase confidencewith respect to the identified set of roadway surface lane markings thevehicle is to follow.
 3. The method of claim 1, wherein identifying aset of roadway surface lane markings comprises detecting at least twocandidate lane markings of a different visual appearance.
 4. The methodof claim 3, wherein detecting at least two candidate lane markings of adifferent visual appearance comprise detecting at least two candidatelane markings that differ in on or more of: intensity of color,reflectivity, presence of lane reflectors, paint cracking, and road tapepeeling.
 5. The method claim 1, wherein accessing sensor data from aplurality of sensors comprises accessing data from two or more of: animage capture device, a lidar system, and a radar system.
 6. A methodfor use at a computer system, the computer system including one or moreprocessors and system memory, the method for automatically controlling avehicle by distinguishing road-surface markings on a roadway, the methodcomprising the one or more processors: accessing sensor data from aplurality of sensors, the plurality of sensors monitoring the roadwaywhere the vehicle is traveling, the plurality of sensors including afirst type of sensor and at least a second different type of sensor, thesensor data indicating road-surface markings, the road-surface markingsincluding intensity of color and reflectivity, the sensor data alsoindicating the location and trajectory of neighboring vehicles;determining bounding boxes from the accessed sensor data, the boundingboxes representing regions of interest for distinguishing road-surfacemarkings and neighboring vehicles; processing the accessed sensor datawithin the bounding boxes to determine that multiple road-surfacemarkings are present; in response to determining that multipleroad-surface markings are present: utilizing the sensor data to identifyroad-surface markings, from among the multiple road-surface markings,the vehicle is to follow; using the location and trajectory ofneighboring vehicles to increase confidence with respect to theidentified road-surface markings; and controlling the vehicle's locationand trajectory on the roadway by following the identified road-surfacemarkings.
 7. The method of claim 6, wherein determining that multipleroad-surface markings are present comprises detecting at least twocandidate lane markings of a different visual appearance.
 8. The methodof claim 7, further comprising, determining a visual appearance of eachof the multiple road-surface markings from one or more of: intensity ofcolor, reflectivity, presence of lane reflectors, paint cracking, androad tape peeling.
 9. The method of claim 6, wherein the sensor datacomes from a plurality of sensors, the plurality of sensors selectedfrom among: lidar systems, image-capture devices, and radar systems. 10.The method of claim 9, further comprising, determining the location andtrajectory of neighboring vehicles from the accessed sensor data. 11.The method of claim 6, wherein accessing sensor data comprises accessingsensor data indicating one or more of speed limit information and stopsign information painted onto the road surface.
 12. The method of claim6, wherein identifying the location and trajectories associated withneighboring vehicles on or near the road comprises identifying vehiclesin the same lane as the vehicle.
 13. The method of claim 6, whereinidentifying the location and trajectories associated with neighboringvehicles on or near the road comprises identifying vehicles in adifferent lane than the vehicle.
 14. The method of claim 6, whereinidentifying the location and trajectories associated with neighboringvehicles comprises identifying vehicles traveling in the same directionas the vehicle.
 15. The method of claim 6, wherein identifying thelocation and trajectories associated with neighboring vehicles comprisesidentifying vehicles traveling in the opposite direction as the vehicle.16. A vehicle comprising: one or more processors; system memory, thesystem memory storing instructions that are executable by the one ormore processors; a plurality of sensors, the plurality of sensorsincluding a first type of sensor and at least a second different type ofsensor, the plurality of sensors monitoring a roadway where the vehicleis traveling; the one or more processors executing the instructionsstored in the system memory to automatically control the vehicle,including the following: access sensor data from the plurality ofsensors, the sensor data indicating road-surface markings theroad-surface markings including intensity of color and reflectivity, thesensor data also indicating the location and trajectory of neighboringvehicles; determine bounding boxes from the accessed sensor data, thebounding boxes representing regions of interest for distinguishingroad-surface markings and neighboring vehicles; process the accessedsensor data within the bounding boxes to determine that multipleroad-surface markings are present; in response to determining thatmultiple road-surface markings are present: utilize the sensor data toidentify road-surface markings, from among the multiple road-surfacemarkings, the vehicle is to follow; use the location and trajectory ofneighboring vehicles to increase confidence with respect to theidentified road-surface markings; and control the vehicle's location andtrajectory on the roadway by following the identified road-surfacemarkings.
 17. The vehicle of claim 16, wherein the one or moreprocessors executing the instructions stored in the system memory tosense sensor data comprise the one or more processors executing theinstructions stored in the system memory to receive at least one type ofdata from one of the vehicle's sensors and at least a second, differenttype of data from a different type of the vehicle's sensors.
 18. Thevehicle of claim 16, further comprising the one or more processorsexecuting the instructions stored in the system memory to query thedifferent types of sensor data for road surface markings.
 19. Thevehicle of claim 16, further comprising the one or more processorsexecuting the instructions stored in the system memory to query thedifferent types of sensor data for neighboring vehicles.
 20. The vehicleof claim 16, further comprising the one or more processors executing theinstructions stored in the system memory to parse the sensor data forregions of interest, the regions of interest being bound by boundingboxes.